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We are investigating on-line model-based test generation from non-deterministic output-observable 
Input/Output Extended Finite State Machine (I/O EFSM) models of Systems Under Test (SUTs). We 
propose a novel constraint-based heuristic approach (Heuristic Reactive Planning Tester (^RPT)) for 
on-line conformance testing non-deterministic SUTs. An indicative feature of ^RPT is the capability 
of making reasonable decisions for achieving the test goals in the on-line testing process by using 
the results of off-line bounded static reachability analysis based on the SUT model and test goal 
specification. We present %RPT in detail and make performance comparison with other existing 
search strategies and approaches on examples with varying complexity. 

1 Introduction 

Model Based Testing (MBT) is one of various test automation approaches. We consider a version of 
MBT where the System Under Test (SUT) is represented by a formal model and treated as a "black- 
box" with an interface. MBT is commonly used to test conformance of the SUT to its model. A SUT 
may be modelled by a non-determistic model because of abstraction, distributed behaviour or freedom 
allowed in the specification. Testing conformance in that case requires on-line testing to react to the 
actual behaviour of the SUT. 

A widespread approach to modeling SUTs for test generation is using either Finite State Machines 
(FSMs) or Extended Finite State Machines (EFSMs) JHJdlllSl. Test generation that includes the input 
data generation from EFSM models has been handled with different methods, including evolutionary 
algrithms [ 5 ] , scenarios [ 1 7 ] and symbolic techniques [15]. The formal symbolic framework and notation 
of conformance for models involving data components is handled in El [HI. On-line methods for test 
generation from non-deterministic models have also been studied by various authors llT^[TTl l9llT5ll. 

Although there is a variety of approaches for test generation from EFSMs, most of the methods are 
not applicable or tend to be inefficient when applied to non-deterministic and industrial-scale systems 
for on-line test generation for specified test goals (e.g., coverage criteria). In this paper we propose 
an Heuristic Reactive Planning Tester (#RPT) to improve the scalability and performance of Reactive 
Planning Tester lTT5l 01 by an heuristic constraint-driven on-line test generation technique. The only 
approaches we are aware of that have comparable goals are presented in ||T1 |6l . The comparison is 
presented in Section [4] 

The integral part of #RPT is an on-line decision-making algorithm responsible for computing the 
stimuli to the SUT based on various constraints emerging from the model of the SUT. This algorithm 
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Figure 1 : Workflow of I/O EFSM based on-line test generation and execution 



draws inspiration from the paradigm of Constraint-Based Local Search (CBLS) [ 10], which has evolved 
into programming language Comet that we use for prototyping. As #RPT is based on on-line decision- 
making, it can also be used in reactive model-based planning in testing (cf. [ 18 ]). This is because #RPT 
computes only one move at a time and is able to react to the observed output of the SUT and the changes 
in test goals on-the-fly. General workflow of I/O EFSM based on-line test generation and execution 
discussed in this paper is depicted in Fig. [TJ The scope of #RPT is highlighted with a dashed rectangle. 

The rest of this paper is structured as follows. In Sect. [2] we outline the relevant background theory 
and preliminaries. Next, in Sect. [3] we describe #RPT in detail. The experimental results are described 
and analyzed in Sect. |4| Finally, Sect. |5]includes the discussion and further work. 



2 Preliminaries 

In this section we introduce the background theory of #RPT. At first, in Sect. |2.1| and |2.2| we define 
the modeling formalism and general testing framework. Next, we give the necessary description of the 



Reactive Planning Tester (RPT) in Sect. 2.3 The background theory is also illustrated with a simple 
three- variable counter example. 



2.1 Input/Output Extended Finite State Machines 

In this paper we assume that the SUT is modelled as an output-observable deterministic or non-deterministic 
I/O EFSM over a first-order theory. For simplicity, the definitions given in this paper use formulas of 
first-order theory of linear integer arithmetic. It is also applicable to other theories where the Satisfiablity 
Modulo Theories (SMT) problem is decidable. 

Definition 1. A constraint over variables X is a first-order formula of the chosen theory (e.g., over 
arithmetic expressions) where variables in X occur as free variables. It is assumed, but not required to 
be quantifier-free for efficiency reasons. 

Definition 2. An I/O EFSM M is a tuple (L,lo,X,D,I,0,G,U,T) where L is a finite set of locations, 
Zo is an initial location, X = X$ U Xi U Xo U Xj r is a disjoint set of finite sets of state, input, output and 
trap variables, D is a constraint over X constraining the domain of variables, I is a finite set of input 
labels that may have an associated set of parameters x\ ,...,x n (xk 6 Xf), O is a finite set of output labels 
that may have an associated set of parameters xi,...,x n (xk £ Xo), G is a finite set of guard conditions 
(constraints), U is a finite set of transition update functions and TcLxIxOxGxU x Lis a finite set 
of transitions. 
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The definition of I/O EFSMs is inspired by UML State-charts and allows intuitive modelling of inter- 
actions between the system and its environment. The main difference with the straightforward semantics 
of the formalisms of Symbolic Transition Systems (STSs) and Input-Output Labelled Transition Systems 
(IOLTSs) [2] is that I/O EFSMs allow transitions to have both input and output labels assigned to them 
simultaneously. It is possible to define the semantics based on STSs such that a transition of I/O-EFSM 
corresponds to two consecutive transitions of STSs. However, we keep the input and output together 
because the presented method deals with interactions as unitary events. Guard conditions and all other 
constraints also implicitly include variable domain constraints D for all variables in X. 

Functions source{t), targetit), guard{t), update{t) and out{l) C T on I/O EFSMs serve as a short- 
hand reference to source and target location, guard condition and update function of a transition t and 
the set of outgoing transitions from location / respectively. 

Definition 3. A state s £ S of I/O EFSM is a pair (l,oc) of a location I £ L and assignment a of state 
variables in X$. 

Therefore, input variables Xj and output variables Xo are not considered to be a part of an I/O EFSM 
state. The values of Xj and Xo are relevant only for the current transition. The input variables that model 
the parameters of input can only occur in the guards and in the right hand sides of updates. Output 
variables that model the parameters of output can only occur in the left hand sides of updates. 

A transition (l,i,o,g,u,l') is enabled and can be taken when an input i is received and guard g 
evaluates to true on the current state and values of the input parameters. The receiving of input i is 
modelled on the logical level by a special input-variable ihabel. We can say that a transition is enabled 
when a formula g AD A ihabel = i evaluates to true. 

Definition 4. An I/O EFSM is said to be non-deterministic, if there exists a state I for which two or more 
guard conditions of transitions in out (I) are non-disjoint and therefore satisfiable using the same input 
and state variables assignment. Transitions t £ out (I) satisfying this criteria are called rival transitions 
and are denoted Rival t . Transitions t' £ out(l) with guard(t') equal to or weaker than guard(t) (i.e., the 
guards are undistinguishable from each other for every assignment of input variables) are perfect rivals 
to t and denoted Rivalf. 

It is further assumed that the SUT is modeled as an output observable I/O EFSM. This means that 
even though in a given state (I, a) multiple rival transitions in out (I) may be taken in response to the 
input, the observed output of the SUT determines the actual move and the next state unambiguously. It 
is possible to relax the condition in expense of increasing the complexity of the on-line computation to 
find the best next move from a set of possible ones. We find this kind of limited non-determinism to be 
practical both for modelling and test generation point of view. 

To further clarify the given notions, Fig. [2] describes an output observable non-deterministic I/O 
EFSM model of a simple thee- variable counter where all variables have domains [0,25]: 
Mi = (L,l ,X,D,I,O,G,U,T), where L = {l ,h,h},X s = {x,y,z},X 1 = {i},X o = 0,X tr = {trapJ 3 }, 
D = (0<x<25A0<y<25A0<z<25A0<i<25A (trapJ 3 = true V trap J 3 = false)), 
I = {ST ART, COUNT, RESET}, O = {TO, Tx, Ty, Tz, T2, T3} 

The model Mi is non-deterministic as it has the following sets of rival transitions: t x with t y , t y with 
(t x ,t z ), t z with (t y ,t2) and ?2 with t z . 

2.2 Modeling Test Goals 

A test goal is a property of the SUT that is intended to be tested. The test goals are modeled as sets of 
traps attached to specific transitions. We classify traps as uncovered, covered or discarded. 



118 



Constraint-Based Heuristic On-line Test Generation from Non-deterministic I/O EFSMs 




Figure 2: I/O EFSM model Mi of a simple three-variable counter 

Definition 5. A trap is a pair (ti,P tr ) where U is a transition and P tr is a constraint on XsUXi. The trap 
(ti,Ptr) is covered when the transition t( has been taken from the state and input where P tr was satisfiable. 

In the following, a trap (ti,P tr ) is denoted with a boolean trap variable tr G Xj r (or sometimes with 
trapJi 6 Xj r )- For a trap (tj,P tr ), the value of the trap variable tr is false if the trap is either uncovered 
(initially all traps) or discarded and becomes true when the trap gets covered (cf. Def. [5]). An uncovered 
trap becomes discarded only when it is determined in the on-line algorithm that it can not be covered. 

The constraint P tr can also include trap variables tr' of other traps (tr / tr') which, in turn, introduces 
a dependency relation between the traps and enables one to talk about specific paths (i.e., sequences of 
trap-labeled transitions). Defining the test goals as sets of traps allows one to use different test strategies 
such as state, transition, path and constrained path coverage lfT3l . Furthermore, we say that a test goal 
is fully satisfied when all its traps have been covered. 

2.3 Reactive Planning Tester 

The Reactive Planning Tester (RPT) is an on-line tester for black-box conformance testing of SUTs 
that are modeled by non-deterministic output-observable I/O EFSMs. As the RPT has been thoroughly 
described in lfl5l . we outline only the aspects relevant to #RPT. 

Due to possible non-determinism, it is not possible to compute inputs for the SUT in advance, i.e., 
off-line. Therefore the workflow of the RPT is further divided into on-line and off-line procedures 
for efficient input generation and computationally hard symbolic analysis. The RPT off-line process 
performs symbolic reachability analysis and generates a system of reachability constraints describing 
the feasible paths (i.e., sequences of transitions) needed to be taken to cover the defined traps. 

For every trap tr the RPT generates the following: 1) A weakest constraint C* ltr on state variables 
X$ and path length J£* t for every location /. Cf tr represents a symbolic state for which there is a path 
with length (i.e., the number of transitions in a path) less or equal to Jzf/* r that covers the trap tr. 2) A 
guarding constraint Cf tr on state and input variables X$UXj for every transition t, which represents a 
symbolic state and input for which the transition t is the initial transition of a shortest path to the trap tr. 
Assignment of false to any of the constraints represents a failure to generate a feasible constraint. 

The reachability constraints are generated by a recursive procedure backwards from the trap until 
one of the following termination conditions is met - (i) a fixpoint is reached, (ii) the constraints have 
been generated for the initial location or (iii) a predefined bounded depth limit is reached. The case (iii) 
is common to location-, transition- or path-wise large systems where it is computationally infeasible to 
generate the constraints until termination condition (i) or (ii) is satisfied. 
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During the on-line procedure, the goal of the RPT is to guide the SUT by using the reachability 
constraints generated off-line to satisfy the test goals. In general, the RPT on-line procedure works as 
follows. First, an uncovered trap with shortest path length S£* tr and satisfiable constraint C* Ur is selected 
in the current state (I, a). Then the assignment of input variables is computed by solving Cf tr (adding 
the negations of the guards of rival transitions where necessary) for transition t in out {I). Finally, the 
tester feeds the inputs to the SUT and observes its output to test conformance, simulates the transition 
and repeats the steps in the target location. 

The RPT on-line algorithm is only applicable when the constraints have been generated for location 
/ the SUT is in and C[ tr is satisfiable in the given state (/, a). If either of these conditions is unsatisfied, 
the generated reachability constraints can not be directly used in this state. To overcome this problem, 
the RPT currently incorporates simple but inefficient random and anti-ant search strategies to make 
randomized intermediate moves when the constraint based input generation is not possible. 

For illustrating the RPT, the reachability constraints generated for the model M\ in Fig. [2] and 
trap (t$,true) (denoted by trap_t3, abbreviated as trj) using the bounded depth 2 are the following: 
(CU.^W <" (f alse -) > ( C U>-^ 3 ) <" (*= 10Aj = 6Az = 2,2), (q^VJ <- (true,!), 
Cf h tn *~ false, Cf xttrj <- true, Cf fn <- (x = 11 Ay = 5 Az = 2), Cf_ ttr3 <- (x = 10 Ay = 6 Az = 1), 
Cf 2 trj <r- (x = 10 A y = 6 A z = 2) and Cf j tr3 <- true. 



3 Heuristic Reactive Planning Tester (#RPT) 

In this section we describe the Heuristic Reactive Planning Tester (^RPT) for I/O EFSM based on-line 
test generation. The aim of #RPT is to improve the scalability and efficiency of on-line test generation. 
#RPT is designed to be used when the RPT on-line algorithm is not directly applicable and is, similarly 
to the RPT, divided into off-line and on-line procedures described in Sect. |3.1|and 3.2 



3.1 Off-line Analysis of I/O EFSMs in #RPT 

In this section we describe the off-line analysis of I/O EFSMs. The goal of this analysis is to provide extra 
input for the proposed on-line decision-making algorithm and to avoid unnecessary repetition of on-line 
computations. The analysis is made after the off-line symbolic reachability analysis in the RPT and prior 
to the on-line test generation and execution. This analysis is based on the I/O EFSM model M and the 
output generated by the RPT off-line process. As a result, distance matrix Dist, search neighbourhood 
Lf rl and sets Tr + , Tr_ of traps are generated as follows. 

The all-pairs shortest distance matrix Dist is computed from the underlying directed control graph 
of the I/O EFSM with Distu = for reflexive transitions and Dist^ j 2 = °° when there is no path from l\ 
to h for all I, l\ , I2 E L. The use of graph based distances between locations is introduced to favour closer 
traps in the on-line decision making algorithm introduced in the next section. 

The search neighbourhood L c is a set indexed both by traps tr £ Xj r and locations I G L of sets 
of closest (control graph based) locations (not necessarily direct neighbours) to / that have reachability 
constraints generated for them for a given trap tr. If the location / G L has reachability constraints gen- 
erated for itself for trap tr, Lf rl includes both / and the next closest set of locations that have reachability 
constraints assigned to them for trap tr. In addition, the locations whose reachability constraints are 
equivalent to or weaker than domain constraints are removed from the search neighbourhood as they 
provide no information to our on-line algorithm. 



120 



Constraint-Based Heuristic On-line Test Generation from Non-deterministic I/O EFSMs 



1: ON_LlNE_ALGORITHM(M,/, a,X ,D,L C ,L T ,Dist , Tabu,C* ,Tr + ,TrJ): 

2: while Tr+ / do 

3: (7Y+,7Y_) <r- On_Line_RPT(Z, a, C*,7Y + ,7Y_) 

4: (MinHea p,Tr + , Tr , L T ) <- 

5: Generate.S olution_C andidates(7, a,X ,D,L C ,L T , Dist , Tabu,C* , 7Y + , 7> ) 

6: besUnove <- CHOOSE_MOST_PROMISING(7, a,X,D,Lf r/ ,Dist, Tabu,C* , Tr + , Tr-) 

7: new. state <— Interact_With_SUT(M, /, cc,bestjnove) 

8: if ne\vstate = () then 

9: return TEST .FAILED 

10: else 

11: (Tabu, I, a) <— newjstate 

12: return TEST FINISHED 

Figure 3: On-line decision-making algorithm of #RPT consisting of four subroutines 
l: ON_LlNE_RPT(/,a,C*,7> + ,7Y_): 

2: while %\ (SAT_Model(X/, a,Cf tr )) for any tr e Tr + then 
3: (I, a) <— RPT_ON_LlNE_ALGORITHM(fr, /, a) 
4: SET_COVERED(fr) ; UPDATE_NEIGHBOURHOOD (rr + , Tr_) 
5 : return (Tr + , Tr_ ) 

Figure 4: Subroutine #1 that uses the RPT on-line algorithm to cover a given trap as soon as reachability 
constraints are satisfied 



Tr + C Xj r and 7Y_ C Xj r are the respective sets of uncovered traps that can be covered and that can 
not be covered from the current state. A function update _neighbourhood(Tr + ,Tr-) is used to update 
the two sets by removing already covered traps from Tr + and moving new coverable traps from Tr_ 
to Tr + . In this paper it is assumed that M has a connected underlying control graph. As a result, the 
partitioning of traps between Tr + and 7Y_ in function update jieighbourhood(Tr + ,TrJ) is based only 
on the dependency ordering of the traps, i.e., which trap variables are used in the constraints of other 
traps. After the off-line analysis has been completed, Tr + U Tr = Tr holds. 

In case the underlying control graph of M not being connected, one could also add a strongly con- 
nected component (SCC) analysis to the algorithm. This could then be used to give higher priority to 
traps in the current SCC to cover them before moving to the next SCC. Moreover, one could also add 
other selection and partitioning criteria but this is left as a further research. 



3.2 On-line Decision-Making Algorithm of #RPT 

In this section we describe the on-line algorithm of #RPT responsible for decision-making during on-line 
test generation in situations when the RPT on-line algorithm is not immediately applicable (cf. discussion 
in Sect. 2.3 1. The #RPT on-line algorithm consists of a top-level algorithm in Fig. |3]and four subroutines 
in Figs. R]l5ll6l[71 They all make use of the I/O EFSM model M, the reachability constraints generated 



by the RPT off-line algorithm and the output of #RPT off-line algorithm discussed in Sect. 3. 1 



In general, the on-line algorithm of #RPT works by making computationally inexpensive operations 
first and then iteratively excludes solution candidates (i.e., possible moves) as the computations become 
more costly until the most promising solution candidate has been selected. Solution candidates are tuples 
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GENERATE_SOLUTION_CANDIDATES(Z, a, X,D,L C ,L T ,Dist,Tabu,C* ,Tr + ,Tr^): 

for all run G {0, 1} do 
for all tr G 7Y + do 

MinHeap' «— 
for all t G ow?(Z) do 

formula <— guard(t) At'eRival, ~^guard(t') AD(X) [update(t) /X] 

formula <— formula A -iTabu tr j 

(b,CCi) <— SAT_MODEL(f,X/,a,/ormwZa) 

if -ifc then continue 

for all l c G Lf rl do 
if (run = 0) V (run = 1 A ~^(target(t) G A source(t) G L^)) then 
dirt <-l+OMWj cr (,) iic 

viol <- v(Cf ctr [update(t)/X][a(x s )/x s ][(Xi(xi)/xi\) 
f <- dist 2 + viol 2 

MinHeap' <— MinHeap' U (t,(Xi,lc,tr,f) 
if MinHeap' = then 
if run = then 

Tabu tr j <r- ; Lf r <- Lf r U I 
else 

SET_DlSCARDED(7r) ; UPDATE_NEIGHBOURHOOD(7Y + , 7Y_) 

else 

MinHeap «— MinHeap [J MinHeap' 
if MinHeap / then 
break 

return (MinHea p , Tr + , 7Y_ , L r ) 



Figure 5: Subroutine #2 that generates solution candidates, excludes excessive ones and orders the re- 
maining by fitness function values 
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Choose Most Promising(/, a,X,D,Lf rl ,Dist, Tabu,C* ,Tr + , 7>_): 

best.f = oo ; bestjnove = 

for up to ./V tuples (t,a' u lc,tr,f) G MinHea p do 

formula <— guard (t) AD(X)[update(t) /X] A^Tabu tr j 

formula formula /\ t i eRiva i t ~^guard(t') 

(b,OCi) <- OPTlMlZE_MODEL(f, X[, a, f ormula,v(C* lc tr [update(t) /X])) 
dist ^l+Dist target{t) k 

viol <r- v(q c tr [update(t)/X][a(x s )/x s ][ai(xi)/xi]) 
f <r- dist 2 + viol 2 
if / < best J then 

bestjnove <— ((Xi,t,lc,tr,f) 
return bestjnove 



Figure 6: Subroutine #3 that selects the most promising solution candidate as a best possible move from 
a given state 
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INTERACT_WITH_SUT(M, I, a, best jnove): 

(cCi,t,lci tr if) bestjnove 
iLabel «- Get JLabel(oc,) 

actual _move 4- Feed _To_SUT(Create_Mess age (/Lafte/, en;))) 
if SutXonforms (M, actual jnove) then 

Tabu tr j <— Tabu tr j\'MAKE_TABU-ELEMEKT(actual jnove, best jiwve) 

(I, a) «- GET_STATE(SlMULATE_MOVE(acfMa/_move)) 

return (Tabu, I, a) 
else 

return () 



Figure 7: Subroutine #4 that creates a message based on the best move, feeds this message to the SUT 
and observes its output 

A,B — logical formulas v(a >= b) = abs(min(0,v(a) — v(b))) 

a,b — arithmetic expressions v(a > b) = abs(min(0,—\ + v(a) — v(b))) 

v(a = b)= abs(v(a) - v(b)) v(a <b)= abs(max(0, 1 + v(a) — v(fe))) 

v(a / b) = v(a < by a > b) v(a <= b) = abs(max(0, v(a) — v(b))) 

v(AAB) = v(A) + v(fi) v(AVfi) = mm(v(A),v(B)) 

Figure 8: Minimal set of computation rules for the violations degree function v 



(t ,a.i,lc,tr,f) consisting of a transition t, input variables assignment a,-, search neighbourhood location 
lc, trap ?r and fitness function value / which is used to measure and compare the quality of solution 
candidates. In this paper, the fitness function consists of the sum of squares of the control-graph based 
distance to the search neighbourhood location lc and the violations degree (cf. Def. [6]> of the reachability 
constraint Cf c tr . The #RPT on-line algorithm also makes use of the RPT on-line algorithm when the 
SUT has been guided to a state where reachability constraints for at least one trap are satisfied making 
the RPT on-line algorithm applicable. The four subroutines that the #RPT on-line algorithm (Fig. [3]> 
consists of are described in the following paragraphs. 

Definition 6. The violations degree of a constraint C is the value of the function v(C) that is inspired by 
fitness function computation in [14]. The minimal set of computation rules for v(C) is given in Fig. [<§] 
The negation of logical formulas is pushed inside and eliminated (if possible) by De Morgan's laws and 
arithmetic equivalences (i.e., v(^(a > b)) = v(a <= b)). 

Subroutine #1 The #RPT on-line algorithm uses the RPT on-line algorithm through the subroutine 
On_Line_RPT(...) outlined in Fig. [4] as soon as the SUT has been guided to a state (/, a) where reach- 
ability constraints Cf tr for some trap tr are satisfied. Then, the RPT on-line algorithm is called using the 
procedure RPT_On_Line_Algorithm(Z, a, tr) to guide the SUT to a state (l', a') covering tr. Next, 
if Cf, tr , is satisfied for any tr 1 in the state (/', a') such that tr / tr', the routine is repeated. 

Subroutine #2 The subroutine Generate_Solutions_Candidates(...) in Fig. |5]is used to gen- 
erate a limited amount of solution candidates by excluding excessive ones. The core idea of this routine is 
to collect solution candidates ordered by the calculated fitness using the min-heap MinHeap. Excessive 
solution candidates are excluded by strengthening the guards of transitions t in out (I) for the satisfiability 
test procedure SAT_Model(£,X/, a, formula) with the negations of guards of rival transitions and tabu 
list Tabu tr i element (11. 6-7). Given the assignment a of state variables, S ATJVl0DEL(f,X/, a, formula) 
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returns a pair (b, a/) of a boolean value indicating whether the logic formula was satisfiable and a model 
for variables in Xj. The algorithm uses a tabu list to avoid converging into local optimums and therefore 
avoid guiding the SUT to infinite loops by keeping the partial history of previous moves. For every trap 
tr and location /, the tabu list element Tabu tr j consists of a disjunction of conjunctions of I/O EFSM tran- 
sition together with input and state variables assignments. These elements explicitly record the moves 
made in the on-line algorithm. Tabu list becomes full if none of the guards of t G out (I), strengthened 
with the negations of tabu list elements, are satisfiable in the given state. The general principles of tabu 
lists and related search strategies can be found from papers by Fred Glover et al. (e.g., 0). 

If no solution candidates for trap tr are collected to MinHeap on the first run, then the guards are 
weakened by emptying the tabu list Tabu tr j. At the same time, the location / is added to L,J r (the set of 
locations I for each trap tr where tabu list Tabu tr j has been emptied). In addition, in the second run, a 
new condition is added (1. 11) to avoid infinite looping. This condition states that if the tabu lists in the 
current location / and target location of transition t G out (I) have been emptied before (i.e., target (t) £ 
L,J r Asource(t) 6 Lj r ) then the move is not permitted. If no solution candidates are found for trap tr after 
the second run due to the new condition, then tr is marked as discarded. Unfortunately this condition 
does not guarantee the complete unreachability of tr but is merely an over- approximation which turns 
out to be strong enough for #RPT. Therefore the notion discarded is used instead of unreachable. 

Subroutine #3 The subroutine Choose_Most_Promising(...) in Fig. [6]is used to choose the most 
promising solution candidate (i.e., the best possible move in a given state). This subroutine compares 
only up to N E {1,2, size(MinHeap)} solution candidates from all the solution candidates collected in 
MinHeap. We allow to vary N to allow different configurations to be used, e.g., for different time require- 
ments. However, the selection cost in this round is higher than before because of the use of constraint 
solving in procedure Optimize. Model(?,X/, a,C,f) (as opposed to SAT test used in Subroutine #2), 
which also optimizes the assignment 05, of input values to minimize the fitness function /. 

Subroutine #4 The subroutine Interact_With_Sut(...) in Fig. [7]is used for interaction with the 
SUT using the most promising solution candidate bestjnove found in Subroutine #3. The message that 
will be fed to the SUT consists of an input label together with possibly empty list of parameters (cf. Def. 
[2]). The input label is obtained from the valuation of the variable ihabel. The input label also determines 
the input variables whose valuation will be sent as input parameters. After feeding this input message to 
the SUT, the subroutine observes the actual move made. If the SUT conforms to the I/O EFSM model, a 
tabu list element Tabu tr .i is updated with the result of M AKE_TABU_ELEMENT(ac?waZ jnove, bestjnove) 
that is a constraint recording the actual or the best move (cf. comments below). Finally, the algorithm 
returns the updated tabu list and the new state corresponding to making actual jnove. 

It has to be noted that if one would only consider actual moves made by the SUT for tabu list element 
construction, then non-determinism of perfect rivals might force the algorithm to loop infinitely. The 
actual jnove and bestjnove need not be the same and the tabu list would not reflect the history correctly. 
Therefore, in our algorithm, we enforce that bestjnove is used for tabu list element construction if 
actual jnove is already present in the tabu list. Instead of this, a more sophisticated bounded fairness 
criteria could be introduced but it has been omitted from this paper due to space restrictions. 

Using %RPT has two possible outcomes. First, testing can be AqcXwqA failed if the observed behavior 
of the SUT does not conform to the given I/O EFSM model. Alternatively, testing is declared finished 
when Tr + becomes empty and no uncovered traps can be added to it. At this point, not all of the test 
goals might be satisfied because some of the traps might still be discarded or uncovered. One should 
then add these traps back to 7>_ , reset the SUT and run the on-line algorithm again from the initial state. 

The decision-making time of this algorithm is dictated by SAT_MODEL and Optimize_Model as 
these are the two most costly operations (in the worst case double exponential to the size of constraints). 
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The size of the constraints depends non-trivially on the structure of the I/O EFSM model, RPT plan- 
ning depth limit and simplifications involved. In #RPT, the number of calls to these operations in each 
state (I, a) is linear to the number of traps, locations in Lf r[ and transitions in out(l). Therefore, the 
performance could be risen by considering different heuristic (sub)methods for the most promising so- 
lution candidate selection to reduce the number of calls made to these operations. The analysis of such 
heuristic algorithms (e.g., simulated annealing or differential evolution) is omitted from this paper. In the 
worst case, #RPT falls back to an anti-ant-like strategy and has similar performance. On the other hand, 
experimental results in the next section give evidence that, on average, #RPT is superior when compared 
to anti-ant by offering significantly better performance and measures to ensure termination. 

We continue with the three-variable counter example M\ (we abbreviate trapJ3 as tr^) to illustrate 
#RPT further. First, we look at #RPT in the initial state (lo,{x <— 0,y <— 0,z 0}) where there is 
only one solution candidate as the only transition in out (Jo) is t\ and the only location in Lf r t is ly. In 
this situation, SATJVlODEL returns us a tuple (true,i = 0) and thus the value of the fitness function is 
/ <— l 2 + 18 2 . On the other hand, Optimize_Model minimizes the value of / and returns (true, i = 10) 
and thus / <- l 2 + 8 2 . 

Secondly, we look at #RPT in the next state (h,{x <— I0,y <— 0,z <— 0}). This time we have four 
transitions t x ,t y ,t z ,t2 in out(l\) and one location l\ in Lf r3 1 . SAT_MODEL first eliminates solution can- 
didates for transitions t x (as z would violate domain constraints) and t% (as the guard is not satisfied). On 
the other hand, both SAT .Model and Optimize _Model return (true, i = 4) and (true, i = 6) for other 
solution candidates corresponding to t y ,t z . As a result, the fitness function values are f y <— l 2 + 8 2 and 
f z <— l 2 + 7 2 , and therefore, the solution candidate corresponding to u is selected as the most promis- 
ing. This process continues until the state (h,{x <— 10, y <— 6,z, «— 2}) is reached where the reachability 
constraint Cf tr is satisfied and the RPT on-line algorithm can be applied to cover trap tr$. 

4 Experimental Results 

In this section we compare different strategies for generating test sequences for specific test goals. In 
particular, we compare the performance of ^RPT with other search strategies such as the RPT off-line 
algorithm |01[T5l and the randomized version of the anti-ant strategy |9]|. As #RPT can be viewed as 
heuristic explicit-state forward reachability analysis rather than the symbolic analyzis done in the RPT, 
we have also chosen an explicit-state model checking tool UPPAAL [7] with modelling language close 
to EFSM for comparison. It gives a comparison between the guided (#RPT) and random (UPPAAL) 
explicit-state forward analysis. Although our method is intended for non-deterministic models, the com- 
parison is easier to make on deterministic models. The following experiments were conducted on a 
64-bit personal computer with 2.4GHz Intel Core 2 Duo CPU and 8GB of DDR3 RAM using prototypes 
written in Comet. 

4.1 Single Trap Test Goals 

In this section we analyse the three- variable counter introduced in Fig. [2] and the Inres Initiator depicted 
in Fig. [9] We consider only test goals containing one trap to give evidence of the performance of #RPT. 

The Inres protocol is a well-known case study model in software testing and verification communi- 
ties. The connection-oriented protocol consists of an Initiator that sets up a connection and sends data 
and a Responder that receives the data and closes the connection. In this paper we consider only the Inres 
Initiator depicted in Fig. [9] which mimics the formalization given in 0]]. 
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Figure 9: I/O EFSM model M2 of Inres Initiator 



Experimental results in Table [T] describe the performance of #RPT on three different test goals. One 
of the goals is defined on the three- variable counter model Mi consisting of trap trapJ^ = (ts,true). The 
other two are defined on the Inres Initiator model M2 - trapj% = (t%,true) and trapj^ = (t$,true). The re- 
spective initial states of I/O EFSM models Mi andM2 are (/q, {x <— 0,y <— 0,z <— 0}) and (l^, {number <— 
0, counter <— 0}). For all three traps, the reachability constraints are generated to depth 2 by the bounded 
RPT off-line algorithm. The results in Table [T] outline the optimal length of paths generated by the RPT 
off-line algorithm and UPPAAL together with algorithm run times. The results also describe the length of 
paths generated by #RPT (completing the path with the RPT on-line algorithm as soon as it is possible) 
and the paths generated by the randomized anti-ant search strategy. 

The experimental results first indicate that #RPT is able to satisfy all three test goals. Moreover, it 
also outperformed the randomized anti-ant strategy significantly in all three cases. The path generated 
by #RPT is optimal in case of trapl% and trapJ$, but differs from the optimal computed by the RPT 
off-line algorithm and UPPAAL in case of trapJj,. The difference is not significant when compared to 
the failure of the anti ant search strategy and is caused by the combination of assignments in update(t\) 



and the optimization of input variables assignment. In Sect. 4.3 we also show that the difference between 
optimal and generated paths stays insignificant in case of larger industrial systems. 

#RPT also performed efficiently time-wise. In all of the three test goals, the average time spent on 
decision making in each state is well below 10ms making #RPT a feasible candidate for industrial on- 
line testing frameworks. In addition, it is clearly visible that the combination of #RPT on-line algorithm 
together with bounded RPT off-line algorithm also time-wise outperformed the RPT (off-line + on-line). 

4.2 Multiple Trap Test Goals 

In this section we analyse the paths generated by #RPT for test goals consisting of multiple traps defined 
on the Inres Initiator model M2 (Fig. [9]>. All of the traps considered here are dependent on preceding traps 
which means the traps can only be covered in the order they are defined. The reachability constraints are 
generated to depth 1 by the bounded RPT off-line algorithm and the initial state is taken from earlier. 
Many of the test goals in Table [2] are inspired and partially taken from HUH. We consider both (i) 
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Table 1 : Experimental results of test goals consisting of one trap showing the lengths of the generated 
paths and algorithm run times 





Mi trapJj 


M2 trapj% 


M2 trapJs 


Complete off-line RPT 








Path length (Work time (s)) 


11 (14.98) 


8 (6.84) 


6 (6.49) 


Bounded off-line RPT 








Depth (Work time (s)) 


2 (2.09) 


2 (4.5) 


2 (4.5) 


*RPT 








Total path length (Work time (s)) 


23 (0.20) 


8 (0.043) 


6 (0.037) 


On-line path length (^RPT + RPT) 


21+2 


6 + 2 


4 + 2 


Time spent in each state (s) 


0.0082 


0.0061 


0.0074 


Randomized anti-ant 








Work time (s) (min/avg/max) 




0.064 / 0.35 / 0.88 


0.017 / 0.11 / 0.21 


Path length (min/avg/max) 




17 / 80 / 193 


6/25/48 


UPPAAL 








Path length (Work time (s)) 


11 (0.53) 


8 (0.50) 


6 (0.49) 



traps that can be covered immediately one after another and (ii) traps that can not. In particular, traps 
of form (ii) force the SUT to be guided through a set of intermediate states before the next trap can be 
covered. Here, the definition of Inres Initiator given in fTJ is used. The definition in [5] differs from the 
former by to and £4 and causes some of the test goals from [5 ] initially consisting of traps of form (i) to 
be actually of the form (ii). 

The experimental results in Table [2] are given as 8 pairs of test goals and corresponding generated 
paths. These test goals consist each of 8 implicitly given and dependently defined traps. They are given 
as a sequence of transitions each of which has an implicit trap defined for it such that the trap constraint 
consist of a conjunction of trap variables of all the preceding traps in the test goal. We do not give 
explicit performance analysis in this section as the average time-wise performance conforms with the 
results from the previous section. 

The test goals 1-3 consist of traps of form (i), which means that every trap can be covered immedi- 
ately after the preceding one. The test goals of form (i) are used to confirm that they are indeed trivial for 
the combination of #RPT and the RPT. Test goals 4 and 5 both contain one trap of form (ii). It is clearly 
visible that although these traps can not be immediately covered after preceding ones, #RPT is able to 
guide the SUT through the necessary intermediate states. The last three test goals 6-8 contain traps 
randomly chosen from all possible traps and therefore contain multiple traps of form (ii). The results 
again confirm that #RPT is able to generate near-optimal paths for the test goals of form (ii). 

4.3 Industrial Scale Telecom Billing System 

In this section we consider an industrial scale telecom billing system. As this example originates from 
industry, we are unfortunately not permitted to depict it explicitly. Instead, we can only give a description 
of the given I/O EFSM by its general characteristics which includes 13 locations and 43 transitions be- 
tween them, 2 input variables having domains [0, 1 1] and [1 , 32000] and 8 state variables having domains 
[0, 1] (1), [0, 1000] (1) and [0,32000] (6). On average, transition guards in this model consist of 20 state 
and input variables connected with logic and arithmetic operations. 
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Table 2: Generated paths (of the EFSM model transitions) for test goals consisting of multiple traps 
defined on the Inres Initiator model. Lists of transitions in parentheses illustrate how the SUT is guided 
through intermediate states to cover the next trap 



No. 


Test goal 


Generated path 


1 


to — h — ?4 — ?7 — h — ?4 — ?5 — ?4 


to — t\ — t4 — tl — t(, — U — ts — t4 


2 


t\ 1 — to — t\ — ?4 — t(, — U — ts — t4 


tl 1 — to — tl — t4 — % — t4 — ?5 — ?4 


3 


to — ti—t\—t\ — t-i — % — t\ — t^ 


to — h — t\ — ?4 — ti — ?6 — U — t$ 


4 


to—t3—to — t\—t4 — t(, — t4 — tj 


to — {tl,tl,tl,tl,ts)—to—ti—t4 — t( l — t4 — ti 


5 


to — tl — h — ?4 — t-j — tj — tj 


k) — tl — t\—t4 — ti — ti — ti — (?9,?8) 


6 


h — h —tn — ts —tu — t2 — tg —t\i 


(to,ti) — (f4 , , , , ?9 , ) — {to,tl,tn)~ 
(to,ti,t<) 1 tt),t<),t9,t( ) ,t4,t5) — (?4,?14) — (to,tl) — 
(h,t4,tg) — (tu,tn) 


7 


ti — 1 10 — h — tj — ?4 — 1 1 1 — t~i — to 


(to,tl) — (ti,t4,tt),tg,tg,t^,tio) — (?0j?l,?4,%) — 

(h3JOih,h,h,h,t3) — (to,ti,t4) — (tu,tn)— 

(k),tl,t4,tl) — (tu,to) 


8 


tl — tu — tl3 — tlO — t5 — t4 — h — til 


(to,h,t2,ti,t2,t3) — tu — (to,h,h3)— 

(to,t2,tl,t4,t9,t9,t9,t9,tio) — 

(to,ti,t4,t9,t9,tg,tg,t( n t4,t5) — ti, — [t9,tg,tg,tg)— 

t%-{to-t n ) 



The test goal whose analysis is given in Table [3] consists of a sequence of traps corresponding to 
exceeding the monthly mobile internet usage limit. As this model is considerably larger both location- 
and state- wise than the previous models Mi and M%, we also use it to compare how different bounds of 
the RPT off-line algorithm affect ^RPT We consider 4 different search depth bounds - 100, 50, 10 and 
2 iterations. The optimal path has length 189 and it is found by the RPT off-line algorithm in 1.3 hours. 

As one might expect, the anti-ant strategy failed to generate a successful path in reasonable time 
due to the significantly large search space. Similarly, UPPAAL also failed to generate a successful trace 
(and therefore also a test sequence) because the size of the search-space caused the explicit-state model 
checking to run out of memory. Moreover, UPPAAL's failure was independent of used configuration, 
e.g., depth-first/breadth-first search, state-space representation and reduction strategies. 

On the other hand, #RPT was able to satisfy the test goal in each of the five cases of different 
RPT search depth bounds. From Table [3] we can first conclude that the generated path indeed depends 
on the RPT search depth bounds. This corresponds to the intuition that #RPT is complementing the 
backward RPT off-line algorithm with a forward on-line algorithm and the farther the RPT generates the 
reachability constraints, the more information they provide to #RPT. Secondly, we can conclude that the 
difference between the generated path and the optimal path does not strictly depend on the I/O EFSM 
size. Although the generated path for the simple counter model Mi in Sect. |4.1| was 2 times longer 
than the optimal, the difference here for bounds 10 to 100 is less than 1.5 times for a significantly larger 
model. Only for depth 2 the generated path is significantly longer than optimal. 

In conclusion, #RPT works efficiently with the given industrial model by generating near-optimal 
paths time-wise efficiently. As the SUT in this example is a relatively large component of an industrial 
system, we are able to give empirical backing to the capability of #RPT for handling components of 
industrial scale systems. Moreover, #RPT is also able to handle larger models when the RPT off-line 
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Table 3: Experimental results on the industrial scale telecom billing system 



Complete off-line RPT 










Path length (Work time (s)) 




189(4644) 




Bounded off-line RPT 










Depth (Work time (s)) 


100 (2120) 


50 (1086) 


10 (95) 


2(16) 


xRPT 










Total path length (Work time (s)) 


230 (6,7) 


255 (17,4) 


275 (17,0) 


1051 (153,4) 


On-line path length (#RPT + RPT) 


130+100 


205 + 50 


265 + 10 


1049 + 2 


Avg. time in each state 


0,051 


0,084 


0,063 


0,146 


Randomized anti-ant 










UPPAAL 





algorithm search depth bounds and #RPT configuration variables are modified accordingly. 

5 Conclusions and Further Work 

The motivation behind this research was to improve the scalability and efficiency of on-line test gen- 
eration from non-deterministic output-observable I/O EFSMs. Although test generation and execution 
from EFSMs has been studied extensively, on-line test generation from non-deterministic models tends 
to confine itself to computationally inexpensive but inefficient strategies such as random search or anti- 
ant. In this paper, we proposed a constraint-based heuristic approach (Heuristic Reactive Planning Tester 
(#RPT)) for I/O EFSM-based on-line test generation that could be used when the RPT on-line algorithm 
Ifl5l |H is not applicable. #RPT is based on reachability constraints and properties of the underlying 
control graph of the I/O EFSM. We compared #RPT with other search strategies such as the RPT ifTBI . 
a randomized version of the anti-ant strategy [9] and also the ones implemented in the model-checking 
tool UPPAAL Q on a three- variable counter, Inres Initiator and an industrial telecom billing system. 

The models considered in this paper are limited to EFSM models over linear arithmetics. This is 
an important extension compared to modelling systems using FSMs, but not all SUTs can be easily 
modelled using only linear arithmetics. All the results are applicable to models over different theories, 
provided that we have a satisfiability solver, optimization procedure and a function for calculating viola- 
tions degree of the formulae of the used theory. 

We have confined ourselves to only dependency based test goal and trap selection criteria and left 
additional analysis as further work. Further research is also needed for the use of #RPT with not con- 
nected, hierarchical and distributed I/O EFSMs. Moreover, further work will include improvements to 
the fitness function computation and stronger trap discarding and ordering conditions. 
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